The present invention is directed to an apparatus and method for the registration of images to physical space by the use of a weighted combination of points and surfaces. The present invention is more particularly directed to the registration of images of patients"" body parts to the actual patients"" body parts for surgery although it is applicable to any use in which a geometrical transformation represents rigid-body motion or approximate rigid-body motion. Such images can be taken by X-ray computed tomography (CT) or the like.
Registration is the determination of a one-to-one mapping or transformation between the coordinates in one space and those in another such that points in the two spaces that correspond to the same anatomic point are mapped to each other. Registration of multimodal images makes it possible to combine different types of structural information [such as X-ray computed tomography (CT) and magnetic resonance (MR) images] and functional information [such as positron emission tomography (PET) and single photon emission tomography (SPECT)] for diagnosis and surgical planning. Registration of images acquired with the same modality at different times allows quantitative comparison of serial data for longitudinal monitoring of disease progression/regression and postoperative follow up.
Registration of preoperative images with the physical space occupied by the patient during surgery is a fundamental step in interactive, image-guided surgery techniques. Surgical navigation systems use the image-to-physical transformation to track in real time the changing position of a surgical probe on a display of the preoperative images. Stereotactic procedures use the transformation to direct a needle (stereotactic biopsy) or energy (stereotactic radiosurgery) to a surgical target (e.g., tumor) located in the images.
Many methods have been used to register medical images. Image-guided stereotactic surgical procedures have been performed since the early 1970""s using stereotactic frame systems. Such systems generally include a reference frame that provides rigid skull fixation using pins or screws and establishes a stereotactic coordinate system in physical space, a method for stereotactic image acquisition, and a system for mechanical direction of a probe or other surgical instrument to a defined intracranial point. Most current systems relate image space to the physical coordinate space established by the reference frame by attaching a localizing system consisting of N-shaped fiducials during image acquisition. Frames permit neurosurgeons to perform biopsies and to resect deep-seated and previously inaccessible lesions.
Frame-based techniques, however, have several limitations. The frames are bulky and may interfere with the surgical exposure. Patients complain about the weight of the frame and the pain associated with its application. The surgeon is typically limited to target points on a linear trajectory. Perhaps most importantly, frame-based stereotactic systems do not provide real-time feedback to the surgeon about anatomic structures encountered in the surgical field.
To address such limitations, a number of frameless stereotactic systems have been developed over the last decade. Of the many frameless methods that have been used to register medical images, it appears that point-based and surface-based techniques are the most useful for image-to-physical registration. There is mounting evidence that voxel-intensity-based methods might be the easiest and most accurate way to perform mono- and multi-modality image-to-image registration, but their use for image-to-physical registration is probably rather limited until higher quality three-dimensional (3-D) intraoperative images become readily available. Point-based registration involves determining the coordinates of corresponding points in different images and/or physical space and computing the geometrical transformation that best aligns those points, generally in a least-squares sense. Many investigators have performed point-based image-to-physical registration using external anatomic landmarks (e.g., internal and external canthi, nasion), skin-affixed markers, or bone-implanted markers; a recent survey cites more than 70 publications devoted to point-based registration. Surface-based registration involves determining corresponding surfaces in different images and/or physical space and computing the geometrical transformation that best matches those surfaces. Researchers have performed surface-based image-to-physical registration using the skin or the outer skull surface. Points and surfaces can be easily and accurately acquired in physical space using 3-D probes (e.g., articulated mechanical, electromagnetic, ultrasonic, optical), stereo video cameras, and/or laser range-finders.
Most previously reported registration techniques that align three-dimensional (3-D) image volumes by matching geometrical features such as points, curves, or surfaces use a single type of feature. Patents have been issued for the idea of point-based registration using bone-implanted markers (e.g., G. S. Allen, xe2x80x9cMethod and Apparatus for Imaging the Anatomy,xe2x80x9d U.S. Pat. No. 5,016,639, May 1991) and surface-based registration (e.g., C. A. Pelizzari and G. T. Y. Chen, xe2x80x9cMeans to Correlate Images from Scans Taken at Different Times Including Means to Determine the Minimum Distances between a Patient Anatomical Contour and a Correlating Surface,xe2x80x9d U.S. Pat. No. 4,977,505, December 1990). An example of a surface commonly used for registration is the skin-air interface.
A technique has been developed to use multiple features simultaneously (C. R. Meyer, G. S. Leichtman, J. A. Brunberg, R. L. Wahl, and L. E. Quint, xe2x80x9cSimultaneous usage of homologous points, lines, and planes for optimal, 3-D, linear registration of multimodality imaging data,xe2x80x9d IEEE Transactions on Medical Imaging, 14: 1-11, 1995). However, that method uses points, lines, and planes and is accordingly less general and less useful than is desired, since lines are special cases of curves and planes are special cases of surfaces.
It will be readily apparent from the above discussion that a need still exists in the art for a method and apparatus for 3-D image registration that provides high accuracy while minimizing the need for invasive techniques and other sources of patient discomfort.
Accordingly, it is an object of the invention to allow accurate 3-D image registration without the need for frames and with only minimal need for markers.
Another object of the invention is to allow 3-D image registration using a combination of points and surfaces.
Yet another object of the invention is to allow 3-D image registration which is not limited to points, lines and planes.
Still another object of the invention is to allow 3-D image registration through weighting of the various features to be registered.
A still further object of the invention is to allow 3-D image registration while avoiding the above-noted deficiencies in the prior art.
To achieve the above and other objects, the present claimed invention is directed to a 3-D image registration method that uses a weighted combination of multiple geometrical features simultaneously. The method uses an algorithm, called a weighted geometrical feature (WGF) registration algorithm, which has the capability of using points, curves (e.g., line segment sets), and surfaces (e.g., triangle sets). The result of the registration is a transformation function which is used in guiding surgical operations. The present invention is further directed to an apparatus for carrying out the algorithm and performing the surgical operations in accordance with the transformation function.
The main idea of the present invention can be summarized briefly as follows:
1) If extrinsic markers (e.g., skin-affixed or bone-implanted markers) are to be used for registration, first attach such markers to the patient in the region of the anatomy to be imaged and registered.
2) Acquire a 3-D image (e.g., CT, MR, PET, SPECT) of the patient.
3) Segment the geometrical features to be used for registration using standard segmentation techniques.
4) Acquire positional information of geometrical features corresponding to those segmented in the above step using 3-D probes (e.g., articulated mechanical, electromagnetic, ultrasonic, optical), stereo video cameras, and/or laser range-finders.
5) Assign weights to the segmented geometrical features.
6) Register the first image with physical space (image-to-physical registration, or IP registration) using the weighted geometrical feature (WGF) registration algorithm.
7) The transformation can be used for various purposes, e.g., tracking the changing position of a surgical probe on a display of the preoperative images, or directing a needle (stereotactic biopsy) or energy (stereotactic radiosurgery) to a surgical target (e.g., tumor) located in the images.
There are two important ideas in the present invention. First, different weights can be assigned to different points or different regions of a surface. That can be done using the present invention with one or more types of geometrical features. For example, the present invention can be used to perform point-based registration, as the WGF algorithm handles point-based and surface-based registration as degenerate cases. Whereas all published work of which Applicants are aware weights all points equally, the present invention allows the user to assign different weights to different points. If each point has a measurement error that is independently random and normally distributed around the true position, then the maximum likelihood estimate of the transformation parameters is obtained by weighting the ith point by 1/"sgr"i2, where "sgr"i2, is the variance of the measurement error of the ith point. In other words, points which are more accurately measured (smaller "sgr"i2) are weighted more heavily (larger 1/"sgr"i2). Similarly, the present invention can be used to perform surface-based registration. It might be possible to increase the accuracy of such registration by weighting surface points using information such as estimated surface point measurement error, surface model segmentation error, and surface curvature.
The second important idea is that registration can be performed using multiple geometrical features simultaneously. That idea has two potential advantages. First, point-based registration requires a minimum of three noncolinear points. The position of a bone-implanted marker can be determined more accurately than that of a skin-affixed marker or an anatomic point landmark. The major disadvantage of using bone-implanted markers is that an invasive procedure is required to implant each marker. By combining surface information, the present invention allows registration to be performed using only one or two markers. Second, a combination of geometrical features can improve both the accuracy and reproducibility of registration. For example, intraoperative surface information is sometimes restricted to a small patch due to surgical conditions (e.g., surgical drapes). Surface-based registration using limited surface information is almost certainly not very accurate. The WGF algorithm can be used to improve registration accuracy in such circumstances by incorporating a single point (in that case a bone-implanted marker, but potentially an anatomical landmark or skin-affixed marker) that effectively serves as an xe2x80x9canchor.xe2x80x9d
As mentioned previously, there is mounting evidence that voxel-intensity-based methods might be the easiest and most accurate way to perform both mono- and multi-modality image-to-image registration. However, their use for image-to-physical registration is probably rather limited until higher quality 3-D intraoperative images become readily available. Thus, for at least the next decade, it is believed that image-to-physical registration will be based on geometrical features such as points and surfaces. Points and surfaces can be easily and accurately acquired in physical space using 3-D probes (e.g., articulated mechanical, electromagnetic, ultrasonic, optical), stereo video cameras, and/or laser range-finders. All image-to-physical registration work of which Applicants are aware has used exclusively points or surfaces.
The WGF registration algorithm, which not only allows the combination of multiple types of geometrical information but also handles point-based and surface-based registration as degenerate cases, could form the foundation of a xe2x80x9cflexiblexe2x80x9d surgical navigation system that allows the surgeon to use what he considers the method most appropriate for an individual clinical situation. For example, if a neurosurgeon wants to remove a superficial meningioma and decides that a 95% TRE (target registration error) of 5-6 mm is sufficient for his operation, he might choose to do a surface-based registration using the skin. If he decides that he needs additional accuracy, he might initially collect skin surface points, perform a skin surface-based registration, use that registration to make a skin flap, collect surface points over the skull that is exposed after making the skin flap, perform a bone-and-skin surface-based registration, and use that more accurate registration to elevate the craniotomy and perform surgery. The present invention would allow the surgeon to also use one or more anatomic landmarks and/or bone-implanted markers in a bone-skin-and-marker registration. In circumstances where the best possible accuracy is required, as may be the case for skull-base surgery performed using an operating microscope with video overlays containing information derived from preoperative images, three or more bone-implanted markers could be used, possibly in combination with the skull surface.
An algorithm has been developed, implemented, and tested. Its usefulness has been demonstrated by using it to perform both image-to-image (CT-MR) and image-to-physical (CT-physical space) registration of head images. Weighting different regions of a surface can improve the accuracy of surface-based registration. Registration using points and a surface can be significantly more accurate than registration using points or a surface.
While a preferred embodiment will be set forth with respect to registration of patients"" heads to the actual patients"" heads for surgery, the invention should not be construed as limited only to that use. For example, rigid-body transformations are also useful in spinal surgery and in orthopedic surgery. Even soft tissues may be sufficiently rigid for the present invention to be used. While the liver is not a rigid structure, liver motion during breathing is often approximately rigid (cranial-caudal translation) and may be sufficiently rigid for a radiosurgery application. For that matter, the present invention can be applied to any surgical or non-surgical use in which a geometrical transformation represents rigid-body motion, exactly or approximately.
Various aspects of the present invention have been described in the following publications, whose disclosures are hereby incorporated by reference in their entireties into the present disclosure:
C. R. Maurer, Jr. et al, xe2x80x9cRegistration of CT and MR brain images using a combination of points and surfaces,xe2x80x9d Medical Imaging 1995: Image Processing, Proc. SPIE 2434: 109-123, 1995;
C. R. Maurer, Jr. et al, xe2x80x9cA method for registration of 3-D images using multiple geometrical features,xe2x80x9d IEEE Transactions on Medical Imaging, 15: 836-849, 1996;
C. R. Maurer, Jr. et al, xe2x80x9cRegistration of head CT images to physical space using multiple geometrical features,xe2x80x9d Medical Imaging 1998: Image-Processing, Proc. SPIE 3338: 72-80, 1998; and
C. R. Maurer, Jr. et al, xe2x80x9cRegistration of head CT images to physical space using a combination of points and surfaces,xe2x80x9d IEEE Transactions on Medical Imaging, 17: 753-761, 1998.